TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting

Transformers have gained popularity in time series forecasting for their ability to capture long-sequence interactions. However, their high memory and computing requirements pose a critical bottleneck for long-term forecasting. To address this, we propose TSMixer, a lightweight neural architecture exclusively composed of multi-layer perceptron (MLP) modules. TSMixer is designed for multivariate forecasting and representation learning on patched time series, providing an efficient alternative to Transformers. Our model draws inspiration from the success of MLP-Mixer models in computer vision. We demonstrate the challenges involved in adapting Vision MLP-Mixer for time series and introduce empirically validated components to enhance accuracy. This includes a novel design paradigm of attaching online reconciliation heads to the MLP-Mixer backbone, for explicitly modeling the time-series properties such as hierarchy and channel-correlations. We also propose a Hybrid channel modeling approach to effectively handle noisy channel interactions and generalization across diverse datasets, a common challenge in existing patch channel-mixing methods. Additionally, a simple gated attention mechanism is introduced in the backbone to prioritize important features. By incorporating these lightweight components, we significantly enhance the learning capability of simple MLP structures, outperforming complex Transformer models with minimal computing usage. Moreover, TSMixer's modular design enables compatibility with both supervised and masked self-supervised learning methods, making it a promising building block for time-series Foundation Models. TSMixer outperforms state-of-the-art MLP and Transformer models in forecasting by a considerable margin of 8-60%. It also outperforms the latest strong benchmarks of Patch-Transformer models (by 1-2%) with a significant reduction in memory and runtime (2-3X).Comment: Accepted in the Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 23), Research Track. Delayed release in arXiv to comply with the conference policies on the double-blind review process. This paper has been submitted to the KDD peer-review process on Feb 02, 202

Hierarchy-guided Model Selection for Time Series Forecasting

Generalizability of time series forecasting models depends on the quality of model selection. Temporal cross validation (TCV) is a standard technique to perform model selection in forecasting tasks. TCV sequentially partitions the training time series into train and validation windows, and performs hyperparameter optmization (HPO) of the forecast model to select the model with the best validation performance. Model selection with TCV often leads to poor test performance when the test data distribution differs from that of the validation data. We propose a novel model selection method, H-Pro that exploits the data hierarchy often associated with a time series dataset. Generally, the aggregated data at the higher levels of the hierarchy show better predictability and more consistency compared to the bottom-level data which is more sparse and (sometimes) intermittent. H-Pro performs the HPO of the lowest-level student model based on the test proxy forecasts obtained from a set of teacher models at higher levels in the hierarchy. The consistency of the teachers' proxy forecasts help select better student models at the lowest-level. We perform extensive empirical studies on multiple datasets to validate the efficacy of the proposed method. H-Pro along with off-the-shelf forecasting models outperform existing state-of-the-art forecasting methods including the winning models of the M5 point-forecasting competition

Automatic leukocyte nucleus segmentation by intuitionistic fuzzy divergence based thresholding

The paper proposes a robust approach to automatic segmentation of leukocyte‟s nucleus from microscopic blood smear images under normal as well as noisy environment by employing a new exponential intuitionistic fuzzy divergence based thresholding technique. The algorithm minimizes the divergence between the actual image and the ideally thresholded image to search for the final threshold. A new divergence formula based on exponential intuitionistic fuzzy entropy has been proposed. Further, to increase its noise handling capacity, a neighborhood-based membership function for the image pixels has been designed. The proposed scheme has been applied on 110 normal and 54 leukemia (chronic myelogenous leukemia) affected blood samples. The nucleus segmentation results have been validated by three expert haematologists. The algorithm achieves an average segmentation accuracy of 98.52% in noise-free environment. It beats the competitor algorithms in terms of several other metrics. The proposed scheme with neighborhood based membership function outperforms the competitor algorithms in terms of segmentation accuracy under noisy environment. It achieves 93.90% and 94.93% accuracies for Speckle and Gaussian noises respectively. The average area under the ROC curves comes out to be 0.9514 in noisy conditions, which proves the robustness of the proposed algorithm

Temporal Dynamics of Workplace Acoustic Scenes: Egocentric Analysis and Prediction

International audienc